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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communicatlon(s) filed on 17 October 2003 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) K Claim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) (3 Claim(s) 1-20 is/are rejected. 

7) \3 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) ^ The specification is objected to by the Examiner. 

10)K The drawing(s) filed on 17 October 2003 is/are: a)D accepted or b)IEI objected to by the Examiner. . 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/MaiI Date. . 

3) ^ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paoer NofsVMail Date 20031017. 20040628 . 100(>GZZ$ 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 7-05) 
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DETAILED ACTION 

Drawings 

1 . The drawings are objected to because Figures 3, 5, and 6 have a typo. 22 
should be labeled "Pointing devices," not "Printing devices." Corrected drawing sheets 
in compliance with 37 CFR 1 .121(d) are required in reply to the Office action to avoid 
abandonment of the application. Any amended replacement drawing sheet should 
include all of the figures appearing on the immediate prior version of the sheet, even if 
only one figure is being amended. The figure or figure number of an amended drawing 
should not be labeled as "amended." If a drawing figure is to be canceled, the 
appropriate figure must be removed from the replacement sheet, and where necessary, 
the remaining figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional replacement 
sheets may be necessary to show the renumbering of the remaining figures. Each 
drawing sheet submitted after the filing date of an application must be labeled in the top 
margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If 
the changes are not accepted by the examiner, the applicant will be notified and 
informed of any required corrective action in the next Office action. The objection to the 
drawings will not be held in abeyance. 

Specification 

2. The use of the trademarks Windows, Acadia, and Pyramid Vision Technologies, 
Inc. has been noted in this application. It should be capitalized wherever it appears and 
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be accompanied by the generic terminology. Please include the trademark sign TM 
when referring to trademarks. 

Although the use of trademarks is permissible in patent applications, the 
proprietary nature of the marks should be respected and every effort made to prevent 
their use in any manner which might adversely affect their validity as trademarks. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 1 , 3, 4, and 8 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Kim et al. (2002/0036649). 

5. With respect to claim 1, Kim et al. discloses a panoramic visualization system 
(see Figure 1), comprising: a plurality of cameras (100), each of which produces image 
data from its field of view, wherein each camera's field of view overlaps (see Figure 1) 
with a neighboring field of view; a pointing device (tracker 150) for supplying view port 
direction information; and a processing system (170) for receiving said view port 
direction information and said image data from said plurality of cameras, said 
processing system for producing view port data from said received image data in 
response to said received view port direction information, wherein said processing 
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system blends said image data from overlapping fields of view to produce panoramic 
view data that represents a panoramic view (110), wherein said view port data 
represents a portion of said panoramic view that is selected by said view port direction 
information; and wherein said processing system corrects the view port data for relative 
positions of said plurality of cameras. 

6. With respect to claims 3 and 4, Kim et al. teaches a display device (head 
mounted display 160) for displaying said view port data. 

7. With respect to claim 8, Kim et al. teaches said processing system corrects said 
view port data for roll, pitch, or yaw (see Abstract). 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 2, 5, 6, 10, and 15-18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kim et al. in view of Foote (2002/01221 13). 

10. With respect to claims 2 and 15, Kim et al. discloses all limitations of the claimed 
invention except that the processing system corrects for lens distortion. It is inherent 
within the structure of a camera that it contains a lens. Foote teaches said processing 
system corrects said view port data for lens distortion of said plurality of cameras (See 
Figure 1 5 to view how the lens distortion of Figure 14 is corrected). It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to have the 
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correction capability of Foote in the processing system of Kim et al. The motivation for 
doing so would be to correct distortion at the edge of the round lens. 

1 1 . With respect to claim 5, Kim et al. discloses the panoramic visualization system 
of claim 1 . Kim et al. does not disclose expressly a control assembly. Foote discloses 
a control assembly that produces control information, wherein said processing system 
produces said first view port data based on said control information (Paragraphs 0145- 
0146 describe a control assembly). It would have been obvious to one of ordinary skill 
in the art at the time the invention was made to add the control assembly of Foote to the 
invention of Kim et al. The motivation for doing so would be to have zoom and other 
capabilities. 

12. With respect to claim 6, Kim et al. discloses said pointing device (tracker 150) is 
selected from a group comprising of a mouse, a head tracker, a touch screen, and a 
joystick. 

13. With respect to claim 10, Kim et al. discloses the panoramic visualization system 
according to claim 1 . Kim et al. does not expressly disclose that the processing system 
employs a vision processing board. Foote discloses a description of a vision processing 
board in Paragraph 0146. It would have been obvious to one of ordinary skill in the art 
at the time the invention was made to use this vision processing board of Foote as the 
processor in the invention of Kim et al. The motivation for doing so would be to speed 
up the computation time. 

14. With respect to claims 16, Kim et al. discloses a display device (head mounted 
display 160) for displaying said view port data. 
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1 5. With respect to claims 1 7 and 1 8, Kim et al. discloses that multiple view port 
direction information (direction of A and B) is obtained, and wherein multiple panoramic 
views (A and B), each selected by associated view port direction information, are 
produced and displayed. 

16. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kim et 
al. in view of Foote as applied to claim 15 above, and further in view of Hansen et al. 
(6,081,606). 

17. Kim et al. as combined discloses all limitations of the claimed invention except for 
the processing automatically tracking a moving object. Hansen et al. discloses 
processing that automatically tracks a moving object (The Abstract states, "Apparatus 
and a concomitant method of identifying the direction of motion within a scene that is 
represented by a sequence of images."). At the time the invention was made, it would 
have been obvious to one of ordinary skill in the art to install the movement tracker 
capabilities of Hansen et al. into the invention of Kim et al. The motivation for doing so 
would be to obtain the regions of interest by identifying and tracking the regions where 
movement is present. 

18. Claims 7 and 11-13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Kim et al. in view of Hansen et al. 

19. With respect to claims 7 and 1 1 , Kim et al. discloses a panoramic visualization 
system, comprising: a plurality of cameras, each of which produces image data from its 
field of view, wherein each field of view overlaps with a neighboring field of view; a first 
pointing device (150 of A) for supplying first view port direction information (A); a second 
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pointing device (150 of B) for supplying second view port direction information (B); and 
a processing system (170) for receiving said first view port direction information, said 
second view port direction information, and said image data from said plurality of 
cameras, said processing system for producing first view port data from said received 
image data in response to said received first view port direction information, said 
processing system further for producing second view port data from said received 
image data in response to said received second view port direction information, wherein 
said processing system blends image data from overlapping fields of view to produce 
panoramic view data that represents a panoramic view (110), wherein said first view 
port data represents a portion of said panoramic view that is selected by said first view 
port direction information, wherein said second view port data represents a portion of 
said panoramic view that is selected by said second view port direction information. 
Kim et al. does not expressly disclose that one of the view ports tracks a moving object. 
Hansen et al. discloses that at least one of the view ports automatically tracks a moving 
object (The Abstract states, "Apparatus and a concomitant method of identifying the 
direction of motion within a scene that is represented by a sequence of images."). It 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to install the movement tracker capabilities of Hansen et al. into the invention of 
Kim et al. The motivation for doing so would be to obtain the regions of interest by 
identifying and tracking the regions where movement is present, which are generally the 
regions of interest. 
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20. With respect to claims 12 and 13, Kim et al. discloses a first display device (HMD 
of A 160) for displaying said first view port data and a second display device (HMD of B 
160) for displaying said second view port data. 

21. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kim et 
al. in view of Hansen et al. as applied to claim 1 1 above, and further in view of Foote. 

22. Kim et al. as combined discloses the panoramic visualization system according 
to claim 1 1 . Kim et al. as combined does not expressly disclose a control assembly. 
Foote discloses a control assembly that produces control information, wherein said 
processing system produces said first view port data based on said control information. 
Paragraphs 0145-0146 describe a control assembly. It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to add the control 
assembly of Foote to the invention of Kim et al. as combined. The motivation for doing 
so would be to have zoom and other capabilities. 

23. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kim et al. 
in view of Tuck (4,772,942). 

24. Kim et al. discloses the panoramic visualization system according to claim 1 . 
Kim et al. does not disclose expressly the cameras are mounted on a vehicle. Tuck 
discloses in the Abstract a plurality of cameras mounted on a moving vehicle. At the 
time the invention was made, it would have been obvious to one of ordinary skill in the 
art to mount the panoramic visualization system of Kim et al. onto a vehicle as taught by 
Tuck. The motivation for doing so would be to expand the use of the system in other 
environments. 



Application/Control Number: 10/688,435 Page 9 

Art Unit: 2851 

25. Claims 20 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over Kim 
et al. in view of Tuck and further in view of Hansen et al. 

26. Kim et al. discloses a system comprising: a plurality of cameras (100), wherein 
each camera produces image data from its field of view, and wherein each camera's 
field of view overlaps (see Figure 1 ) with a neighboring field of view; a pointing device 
(150) for supplying view port direction information; and a processing system (170) for 
receiving said view port direction information and said image data from plurality of 
cameras, said processing system for producing view port data from said received image 
data in response to said received view port direction information, wherein said 
processing system blends said image data from overlapping fields of view to produce 
panoramic view data that represents a panoramic view (110), and wherein said view 
port data represents a portion of said panoramic view that is selected by said view port 
direction information. Kim et al. does not disclose expressly a vehicle body to mount 
the plurality of cameras, nor the processing system tracking an object. 

Tuck discloses in the Abstract a plurality of cameras mounted on a moving 
vehicle. At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to mount the panoramic visualization system of Kim et al. onto a 
vehicle as taught by Tuck. The motivation for doing so would be to expand the use of 
the system in other environments. 

Hansen et al. discloses that the processing system automatically tracks a moving 
object (The Abstract states, "Apparatus and a concomitant method of identifying the 
direction of motion within a scene that is represented by a sequence of images."). It 
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would have been obvious to one of ordinary skill in the art at the time the invention was 
made to install the movement tracker capabilities of Hansen et al. into the invention of 
Kim et al. as combined. The motivation for doing so would be to obtain the regions of 
interest by identifying and tracking the regions where movement is present. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael Liu whose telephone number is (571) 272-9019. 
The examiner can normally be reached on Monday through Friday 9 am - 5 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Judy Nguyen can be reached on (571) 272-2258. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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